Set below is some relevant background relating to the invention. Additional background is found in the parent application, U.S. patent application Ser. No. 10/701,361, and is incorporated by reference herein.
1. Diagnostics
1.1 General Diagnostics
When a vehicle component begins to fail, the repair cost is frequently minimal if the impending failure of the component is caught early, but increases as the repair is delayed. Sometimes, if a component in need of repair is not caught in a timely manner, the component, and particularly the impending failure thereof, can cause other components of the vehicle to deteriorate. One example is where the water pump fails gradually until the vehicle overheats and blows a head gasket. Another example is when a tire gradually loses air until it heats up, fails and causes an accident. It is desirable, therefore, to determine that a vehicle component is about to fail as early as possible so as to minimize the probability of a breakdown and the resulting repair costs.
There are various gages on an automobile which alert the driver to various vehicle problems. For example, if the oil pressure drops below some predetermined level, the driver is warned to stop his vehicle immediately. Similarly, if the coolant temperature exceeds some predetermined value, the driver is also warned to take immediate corrective action. In these cases, the warning often comes too late as most vehicle gages alert the driver after he or she can conveniently solve the problem. Thus, what is needed is a component failure warning system that alerts the driver to the impending failure of a component sufficiently in advance of the time when the problem gets to a catastrophic point.
Some astute drivers can sense changes in the performance of their vehicle and correctly diagnose that a problem with a component is about to occur. Other drivers can sense that their vehicle is performing differently but they don't know why or when a component will fail or how serious that failure will be, or possibly even what specific component is the cause of the difference in performance. The invention disclosed herein will, in most cases, solve this problem by predicting component failures in time to permit maintenance and thus prevent vehicle breakdowns.
Presently, automobile sensors in use are based on specific predetermined or set levels, such as the coolant temperature or oil pressure, whereby an increase above the set level or a decrease below the set level will activate the sensor, rather than being based on changes in this level over time. The rate at which coolant heats up, for example, can be an important clue that some component in the cooling system is about to fail. There are no systems currently on automobiles to monitor the numerous vehicle components over time and to compare component performance with normal performance. Nowhere in the vehicle is the vibration signal of a normally operating front wheel stored, for example, or for that matter, any normal signal from any other vehicle component. Additionally, there is no system currently existing on a vehicle to look for erratic behavior of a vehicle component and to warn the driver or the dealer that a component is misbehaving and is therefore likely to fail in the very near future.
Basically, the operating of an automobile should be a process not a project. A purpose of this invention is to eliminate breakdowns through identifying potential component failures before they occur so that they can be repaired in a timely manner. Another purpose is to notify the operator and a service facility of the pending failure so that it can be prevented.
Sometimes, when a component fails, a catastrophic accident results. In the Firestone tire case, for example, over 100 people were killed when a tire of a Ford Explorer blew out which caused the Ford Explorer to rollover. Similarly, other component failures can lead to loss of control of the vehicle and a subsequent accident. It is thus important to accurately forecast that such an event will take place but furthermore, for those cases where the event takes place suddenly without warning, it is also important to diagnose the state of the entire vehicle, which in some cases can lead to automatic corrective action to prevent unstable vehicle motion or rollovers resulting in an accident.
Finally, an accurate diagnostic system for the entire vehicle can determine much more accurately the severity of an automobile crash once it has begun by knowing where the accident is taking place on the vehicle (e.g., the part of or location on the vehicle which is being impacted by an object) and what is colliding with the vehicle based on a knowledge of the force deflection characteristics of the vehicle at that location. Therefore, in addition to a component diagnostic, the teachings of this invention also provide a diagnostic system for the entire vehicle prior to and during accidents. In particular, this invention is concerned with the simultaneous monitoring of multiple sensors on the vehicle so that the best possible determination of the state of the vehicle can be determined. Current crash sensors operate independently or at most one sensor may influence the threshold at which another sensor triggers a deployable restraint as taught in the current assignee's U.S. patent application Ser. No. 10/638,743 filed Aug. 11, 2003 and related patents and pending applications. In the teachings of this invention, two or more sensors, frequently accelerometers, are monitored simultaneously and the outputs of these multiple sensors can be combined continuously in making the crash severity analysis.
U.S. Pat. No. 5,754,965 (Hagenbuch) describes an apparatus for diagnosing the state of health of a construction vehicle and providing the operator of the vehicle with a substantially real-time indication of the efficiency of the vehicle in performing as assigned task with respect to a predetermined goal. A processor in the vehicle monitors sensors that provide information regarding the state of health of the vehicle and the amount of work the vehicle has done. The processor records information that describes events leading up to the occurrence of an anomaly for later analysis. The sensors are also used to prompt the operator to operate the vehicle at optimum efficiency. The system of this patent does not predict or warn the operator or the home base of a pending problem.
Asami et al. (U.S. Pat. No. 4,817,418) is directed to a failure diagnosis system for a vehicle including a failure display means for displaying failure information to a driver. This system only reports failures after they have occurred and does not predict them.
Tiernan et al. (U.S. Pat. No. 5,313,407) is directed, inter alia, to a system for providing an exhaust active noise control system, i.e., an electronic muffler system, including an input microphone 60 which senses exhaust noise at a first location 61 in an exhaust duct 58. An engine has exhaust manifolds 56,57 feeding exhaust air to the exhaust duct 58. The exhaust noise sensed by the microphone 60 is processed to obtain an output from an output speaker 65 arranged downstream of the input microphone 61 in the exhaust path in order to cancel the noise in the exhaust duct 58. No attempt is made to diagnose system faults nor predict them.
Haramaty et al. (U.S. Pat. No. 5,406,502) describes a system that monitors a machine in a factory and notifies maintenance personnel remote from the machine (not the machine operator) that maintenance should be scheduled at a time when the machine is not in use. Haramaty et al. does not expressly relate to vehicular applications.
NASA Technical Support Package MFS-26529 “Engine Monitoring Based on Normalized Vibration Spectra”, describes a technique for diagnosing engine health using a neural network based system but does not suggest that this system can or should be used on land vehicles.
A paper “Using acoustic emission signals for monitoring of production processes” by H. K. Tonshoff et al. also provides a good description of how acoustic signals can be used to predict the state of machine tools and is incorporated by reference herein in its entirety. Again no suggestion is made that this can be used for diagnosing components of land vehicles.
1.2 Pattern Recognition
Marko et al. (U.S. Pat. No. 5,041,976) is directed to a diagnostic system using pattern recognition for electronic automotive control systems and particularly for diagnosing faults in the engine of a motor vehicle after they have occurred. For example, Marko et al. is interested in determining cylinder specific faults after the cylinder is operating abnormally. More specifically, Marko et al. is directed to detecting a fault in a vehicular electromechanical system directly, i.e., by means of the measurement of parameters of sensors which are designed to be affected only by that system, and after that fault has already manifested itself in the system. In order to form the fault detecting system, the parameters from these sensors are input to a pattern recognition system for training thereof. Then, known faults are introduced and the parameters from the sensors are input into the pattern recognition system with an indicia of the known fault. Thus, during subsequent operation, the pattern recognition system can determine the fault of the electromechanical system based on the parameters of the sensors, assuming that the fault was “trained” into the pattern recognition system and has already occurred.
When the electromechanical system is an engine, the parameters input into the pattern recognition system for training thereof, and used for fault detection during operation, all relate to the engine. In other words, each parameter will be affected by the operation of the engine and depend thereon and changes in the operation of the engine will alter the parameter, e.g., the manifold absolute pressure is an indication of the airflow into the engine. In this case, the signal from the manifold absolute pressure sensor may be indicative of a fault in the intake of air into the engine, e.g., the engine is drawing in too much or too little air, and is thus affected by the operation of the engine. Similarly, the mass air flow is the airflow into the engine and is an alternative to the manifold absolute pressure. It is thus a parameter that is directly associated with, related to and dependent on the engine. The exhaust gas oxygen sensor is also affected by the operation of the engine, and thus directly associated therewith, since during normal operation, the mixture of the exhaust gas is neither rich or lean whereas during abnormal engine operation, the sensor will detect an abrupt change indicative of the mixture being too rich or too lean.
Thus, the system of Marko et al. is based on the measurement of sensors which affect or are affected by, i.e., are directly associated with, the operation of the electromechanical system for which faults are to be detected. However, the system of Marko et al. does not detect faults in the sensors that are conducting the measurements, e.g., a fault in the exhaust gas oxygen sensor, or faults that are only developing but have not yet manifested themselves or faults in other systems. Rather, the sensors are used to detect a fault in the system after it has occurred. Marko does not attempt to forecast or predict that a fault will occur.
Aside from the references above of assignee's patents and patent applications and the one example of an engine control system, pattern recognition has not been applied to the diagnosis of any faults on a vehicle. In the referenced examples, the engine controller for example, only sensors directly associated with the component have been used. No attempt has been made to forecast that a failure will occur and no system has been disclosed other than by the assignee for transmitting such diagnostic information to a site off of the vehicle.
2.0 Telematics
Every automobile driver fears that his or her vehicle will break down at some unfortunate time, e.g., when he or she is traveling at night, during rush hour, or on a long trip away from home. To help alleviate that fear, certain luxury automobile manufacturers provide roadside service in the event of a breakdown. Nevertheless, unless the vehicle is equipped with OnStar® or an equivalent service, the vehicle driver must still be able to get to a telephone to call for service. It is also a fact that many people purchase a new automobile out of fear of a breakdown with their current vehicle. The inventions described herein are primarily concerned with preventing breakdowns and with minimizing maintenance costs by predicting component failure that would lead to such a breakdown before it occurs.
Another important aspect disclosed in the Breed et al. patents relates to the operation of the cellular communications system in conjunction with the vehicle interior monitoring system. Vehicles can be provided with a standard cellular phone as well as the Global Positioning System (GPS), an automobile navigation or location system with an optional connection to a manned assistance facility. In the event of an accident, the phone may automatically call 911 for emergency assistance and report the exact position of the vehicle. If the vehicle also has a system as described below for monitoring each seat location, the number and perhaps the condition of the occupants could also be reported. In that way, the emergency service (EMS) would know what equipment and how many ambulances to send to the accident site. Moreover, a communication channel can be opened between the vehicle and a monitoring facility/emergency response facility or personnel to determine how badly people are injured, the number of occupants in the vehicle, and to enable directions to be provided to the occupant(s) of the vehicle to assist in any necessary first aid prior to arrival of the emergency assistance personnel.
Communications between a vehicle and a remote assistance facility are also important for the purpose of diagnosing problems with the vehicle and forecasting problems with the vehicle, called prognostics. Motor vehicles contain complex mechanical systems that are monitored and regulated by computer systems such as electronic control units (ECUs) and the like. Such ECUs monitor various components of the vehicle including engine performance, carburetion, speed/acceleration control, transmission, exhaust gas recirculation (EGR), braking systems, etc. However, vehicles perform such monitoring typically only for the vehicle driver and without communication of any impending results, problems and/or vehicle malfunction to a remote site for trouble-shooting, diagnosis or tracking for data mining.
In the past, systems that provide for remote monitoring did not provide for automated analysis and communication of problems or potential problems and recommendations to the driver. As a result, the vehicle driver or user is often left stranded, or irreparable damage occurs to the vehicle as a result of neglect or driving the vehicle without the user knowing the vehicle is malfunctioning until it is too late, such as low oil level and a malfunctioning warning light, fan belt about to fail, failing radiator hose etc.
U.S. Pat. No. 5,400,018 (Scholl et al.) describes a system for relaying raw sensor output from an off road work site relating to the status of a vehicle to a remote location over a communications data link. The information consists of fault codes generated by sensors and electronic control modules indicating that a failure has occurred rather than forecasting a failure. The vehicle does not include a system for performing diagnosis. Rather, the raw sensor data is processed at an off-vehicle location in order to arrive at a diagnosis of the vehicle's operating condition. Bi-directional communications are described in that a request for additional information can be sent to the vehicle from the remote location with the vehicle responding and providing the requested information but no such communication takes place with the vehicle operator and not of an operator of a vehicle traveling on a road. Also, Scholl et al. does not teach the diagnostics of the problem or potential problem on the vehicle itself nor does it teach the automatic diagnostics or any prognostics. In Scholl et al. the determination of the problem occurs at the remote site by human technicians.
U.S. Pat. No. 5,955,942 (Slifkin et al.) describes a method for monitoring events in vehicles in which electrical outputs representative of events in the vehicle are produced, the characteristics of one event are compared with the characteristics of other events accumulated over a given period of time and departures or variations of a given extent from the other characteristics are determined as an indication of a significant event. A warning is sent in response to the indication, including the position of the vehicle as determined by a global positioning system on the vehicle. For example, for use with a railroad car, a microprocessor responds to outputs of an accelerometer by comparing acceleration characteristics of one impact with accumulated acceleration characteristics of other impacts and determines departures of a given magnitude from the other characteristics as a failure indication which gives rise of a warning.
Of course there are many areas of the country where cell phone reception is not available and thus a system that relies on the availability of such a system for diagnostics will not always be available and thus has a significant failure mode. Furthermore, it would be difficult if not impossible for such a location to have all of the information to diagnose problems with all vehicle models that are on the road and to be able to retrieve that information and act on raw data on a continuous basis to keep track of whether all vehicles on the roadways are operating properly and to forecast all potential problems with each vehicle. Thus, this function must be resident on the vehicle. Additionally is a human operator is required then the system quickly becomes unmanageable.
3.0 Definitions
As used herein, a diagnosis of the “state of the vehicle” means a diagnosis of the condition of the vehicle with respect to its stability and proper running and operating condition. Thus, the state of the vehicle could be normal when the vehicle is operating properly on a highway or abnormal when, for example, the vehicle is experiencing excessive angular inclination (e.g., two wheels are off the ground and the vehicle is about to rollover), the vehicle is experiencing a crash, the vehicle is skidding, and other similar situations. A diagnosis of the state of the vehicle could also be an indication that one of the parts of the vehicle, e.g., a component, system or subsystem, is operating abnormally.
As used herein, a “part” of the vehicle includes any component, sensor, system or subsystem of the vehicle such as the steering system, braking system, throttle system, navigation system, airbag system, seatbelt retractor, air bag inflation valve, air bag inflation controller and airbag vent valve, as well as those listed below in the definitions of “component” and “sensor”.
As used herein, a “sensor system” includes any of the sensors listed below in the definition of “sensor” as well as any type of component or assembly of components which detect, sense or measure something.
The term “vehicle” shall mean any means for transporting or carrying something including automobiles, trucks, vans, containers, trailers, boats, railroad cars and engines.
The term “gage” as used herein interchangeably with the terms “gauge”, “sensor” and “sensing device”.
The following additional terms will be used in the description of the invention and for the sake of clarity are defined here.
The “A-pillar” of a vehicle and specifically of an automobile is defined as the first roof supporting pillar from the front of the vehicle and usually supports the front door. It is also known as the hinge pillar.
The “B-Pillar” is the next roof support pillar rearward from the A-Pillar.
The “C-Pillar” is the final roof support usually at or behind the rear seats.
The windshield header as used herein includes the space above the front windshield including the first few inches of the roof. The headliner is the roof interior cover that extends back from the header.
The term “squib” represents the entire class of electrically initiated pyrotechnic devices capable of releasing sufficient energy to cause a vehicle window to break, for example. It is also used to represent the mechanism which starts the burning of an initiator which in turn ignites the propellant within an inflator.
The term “airbag module” generally connotes a unit having at least one airbag, gas generator means for producing a gas, attachment or coupling means for attaching the airbag(s) to and in fluid communication with the gas generator means so that gas is directed from the gas generator means into the airbag(s) to inflate the same, initiation means for initiating the gas generator means in response to a crash of the vehicle for which deployment of the airbag is desired and means for attaching or connecting the unit to the vehicle in a position in which the deploying airbag(s) will be effective in the passenger compartment of the vehicle. In the instant invention, the airbag module may also include occupant sensing components, diagnostic and power supply electronics and componentry which are either within or proximate to the module housing.
The term “occupant protection device” or “occupant restraint device” as used herein generally includes any type of device which is deployable in the event of a crash involving the vehicle for the purpose of protecting an occupant from the effects of the crash and/or minimizing the potential injury to the occupant. Occupant restraint or protection devices thus include frontal airbags, side airbags, seatbelt tensioners, knee bolsters, side curtain airbags, externally deployable airbags and the like.
“Pattern recognition” as used herein will generally mean any system which processes a signal that is generated by an object (e.g., representative of a pattern of returned or received impulses, waves or other physical property specific to and/or characteristic of and/or representative of that object) or is modified by interacting with an object, in order to determine to which one of a set of classes that the object belongs. Such a system might determine only that the object is or is not a member of one specified class, or it might attempt to assign the object to one of a larger set of specified classes, or find that it is not a member of any of the classes in the set. The signals processed are generally a series of electrical signals coming from transducers that are sensitive to acoustic (ultrasonic) or electromagnetic radiation (e.g., visible light, infrared radiation, capacitance or electric and/or magnetic fields), although other sources of information are frequently included. Pattern recognition systems generally involve the creation of a set of rules that permit the pattern to be recognized. These rules can be created by fuzzy logic systems, statistical correlations, or through sensor fusion methodologies as well as by trained pattern recognition systems such as neural networks, combination neural networks, cellular neural networks or support vector machines.
A trainable or a trained pattern recognition system as used herein generally means a pattern recognition system that is taught to recognize various patterns constituted within the signals by subjecting the system to a variety of examples. The most successful such system is the neural network used either singly or as a combination of neural networks. Thus, to generate the pattern recognition algorithm, test data is first obtained which constitutes a plurality of sets of returned waves, or wave patterns, or other information radiated or obtained from an object (or from the space in which the object will be situated in the passenger compartment, i.e., the space above the seat) and an indication of the identify of that object. A number of different objects are tested to obtain the unique patterns from each object. As such, the algorithm is generated, and stored in a computer processor, and which can later be applied to provide the identity of an object based on the wave pattern being received during use by a receiver connected to the processor and other information. For the purposes here, the identity of an object sometimes applies to not only the object itself but also to its location and/or orientation in the passenger compartment. For example, a rear facing child seat is a different object than a forward facing child seat and an out-of-position adult can be a different object than a normally seated adult. Not all pattern recognition systems are trained systems and not all trained systems are neural networks. Other pattern recognition systems are based on fuzzy logic, sensor fusion, Kalman filters, correlation as well as linear and non-linear regression. Still other pattern recognition systems are hybrids of more than one system such as neural-fuzzy systems.
The use of pattern recognition, or more particularly how it is used, is important to the instant invention. In the above-cited prior art, except in that assigned to the current assignee, pattern recognition which is based on training, as exemplified through the use of neural networks, is not mentioned for use in monitoring the interior passenger compartment or exterior environments of the vehicle in all of the aspects of the invention disclosed herein. Thus, the methods used to adapt such systems to a vehicle are also not mentioned.
A pattern recognition algorithm will thus generally mean an algorithm applying or obtained using any type of pattern recognition system, e.g., a neural network, sensor fusion, fuzzy logic, etc.
To “identify” as used herein will generally mean to determine that the object belongs to a particular set or class. The class may be one containing, for example, all rear facing child seats, one containing all human occupants, or all human occupants not sitting in a rear facing child seat, or all humans in a certain height or weight range depending on the purpose of the system. In the case where a particular person is to be recognized, the set or class will contain only a single element, i.e., the person to be recognized.
A “combination neural network” as used herein will generally apply to any combination of two or more neural networks that are either connected together or that analyze all or a portion of the input data. A combination neural network can be used to divide up tasks in solving a particular occupant problem. For example, one neural network can be used to identify an object occupying a passenger compartment of an automobile and a second neural network can be used to determine the position of the object or its location with respect to the airbag, for example, within the passenger compartment. In another case, one neural network can be used merely to determine whether the data is similar to data upon which a main neural network has been trained or whether there is something radically different about this data and therefore that the data should not be analyzed. Combination neural networks can sometimes be implemented as cellular neural networks.
Preferred embodiments of the invention are described below and unless specifically noted, it is the applicants' intention that the words and phrases in the specification and claims be given the ordinary and accustomed meaning to those of ordinary skill in the applicable art(s). If the applicant intends any other meaning, he will specifically state he is applying a special meaning to a word or phrase.
Likewise, applicants' use of the word “function” here is not intended to indicate that the applicants seek to invoke the special provisions of 35 U.S.C. §112, sixth paragraph, to define their invention. To the contrary, if applicants wish to invoke the provisions of 35 U.S.C. §112, sixth paragraph, to define their invention, they will specifically set forth in the claims the phrases “means for” or “step for” and a function, without also reciting in that phrase any structure, material or act in support of the function. Moreover, even if applicants invoke the provisions of 35 U.S.C. §112, sixth paragraph, to define their invention, it is the applicants' intention that their inventions not be limited to the specific structure, material or acts that are described in the preferred embodiments herein. Rather, if applicants claim their inventions by specifically invoking the provisions of 35 U.S.C. §112, sixth paragraph, it is nonetheless their intention to cover and include any and all structure, materials or acts that perform the claimed function, along with any and all known or later developed equivalent structures, materials or acts for performing the claimed function.